OCR-Free Document Understanding Transformer

Understanding document images (e.g., invoices) is a core but challenging task since it requires complex functions such as reading text and holistic understanding of the document. Current Visual Document (VDU) methods outsource to off-the-shelf Optical Character Recognition (OCR) engines focus on with OCR outputs. Although OCR-based approaches have shown promising performance, they suffer from 1) high computational costs for using OCR; 2) inflexibility models languages or types documents; 3) error propagation subsequent process. To address these issues, in this paper, we introduce novel OCR-free VDU model named Donut, which stands transformer. As first step research, propose simple architecture (i.e., Transformer) pre-training objective cross-entropy loss). Donut conceptually yet effective. Through extensive experiments analyses, show model, achieves state-of-the-art performances various tasks terms both speed accuracy. In addition, offer synthetic data generator that helps be flexible domains. The code, trained are available at https://github.com/clovaai/donut .